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ABSTRACT 


A  sinulaticn  model  for  the  evaluation  of  program 
structure  and  error  detection  has  been  applied  tc  the 
analysis  of  selected  parts  of  NTDS  programs.  The 
simulation  results  were  used  tc  establish  the 
relationship  between  program  structure  and  measures  of 
program  complexity.  This  information  wculd  be  used 
for  the  design  and  testing  cf  software. 
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INTRODUCTION 


When  is  a  program  considered  to  be  trivial?  One  answer 
to  this  guesticn  heard  very  often  is  "When  it  contains  no 
bugs".  although  this  statement  might  be  questionable ,  the 
converse  is  true,  as  there  are  few  nontrivial  programs  that 
do  net  certain  bugs.  As  the  author  of  a  critical  and 
fundamental  study  of  program  design  states:  "...These  tugs 
can  never  be  completely  exorcised  in  any  program  over  some 
critical  decree  cf  complexity.  Six  months  or  even  seven 
years  after  'final  debugging1  errors  crop  up  inevitably  in 
the  best  cf  programs. "[ 4  ].  This  is  a  fact  one  has  tc  live 
with,  and  there  are  .only  two  things  one  can  do  about  it: 
First  tc  reduce  the  possibilities  for  bugs  by  careful  design 
and  use  cf  medem  programming  techniques,  second  tc  devise 
careful  testing  techniques  to  detect  and  locate  the  bugs 
still  remaining  in  tie  program. 


Fig.  1  shews  the  relationship  between  hardware  and 
software  ccst  in  the  U.S.  during  the  pericd  from  1955  to 
19£5.  Cue  to  the  fact  that  the  software  ccst  continues  to 
rise  and  that  about  50S  of  this  cost  is  fcr  testing  and 
integration  cf  a  system  [7],  it  is  important  to  obtain  a 
realistic  assessment  of  how  much  effort  has  tc  be  spent  to 
test  the  newly  designed  program  based  on  its  size,  structure 
and  characteristics.  If  one  is  able  to  determine  in  the 
design  stage  the  test  possible  structure  with  respect  to  the 
error  detection  capabilities,  then  bugs  can  be  avoided  and 
testing  will  te  reduced.  Also  early  in  the  development  cf  a 
project  a  realistic  allocation  of  coding  and  testing 
resources  could  be  made. 
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In  order  to  address  these  problems,  a  Software  Error 
Detection  Simulation  Model  has  been  developed  [7,10].  This 
model  was  was  used  to  identify  program  complexity  measures 
which  were  correlated  with  error  detection.  Naval  Tactical 
Data  System  jiicgrams  were  used  for  this  purpose. 

The  structures  of  these  NTDS -programs  have  been  analyzed 
(see  Chapter  "VI)  and  put  into  the  form  of  directed  graphs. 


The  date  gained  from  the  directed  graph  representation 
were  used  as  inputs  for  the  Error  Detection  Simulation 
Model.  The  results  gained  and  the  conclusions  and 
recemmendatiens  drain  from  these  results  are  shown  in 
Chapter  vll .  For  reasons  of  security,  the  programs  or  the 
parts  of  then  are  not  identified  by  names.  Instead,  a 
sequential  rumber  scheme  for  identifying  the  prograirs  has 
been  employee. 


This  work  is  part  of  a  research  effort  sponsored  by  tne 
NA£C  to  get  software  evaluation  aids  which  provide  an 
economical  assessment  of  the  design  and  testing  effort 
needed  for  the  development  of  avionics  and  ether  complex 
software  prefects. 

Eecause  it  is  felt  that  efforts  in  testing  acd  in 
debugging  can  be  mere  successful  if  cne  employs  ncdeirn 
technigues  in  the  production  of  programs,  an  introductory 
chapter  shows  the  relevance  of  modern  programming  technigues 
to  the  problem  of  program  testing  and  maintecance. 


II.   DEFINITIONS 


Iher€  was  originally  a  lack  of  commonly  used  definitions 
fcr  program  testing.  Only  recently  has  a  "definitional 
framework"  eierged  and  very  good  program  testing  definitions 
are  found  in  Ref.  8,  pg.  7  -  14.  In  order  to  be  consistent 
and  to  specify  the  meaning  of  keywords  within  this  thesis, 
the  fcllc*inc  definitions  have  been  adopted: 

1  •      Iiccjiam    Structure 

The  structure  of  a  program  is  a  description   of  the 

underlying  logic  and  data  flow  as  represented  in  the 

fcrm  cf  a  directed  graph  with  its  set  of  nodes  and 
edges  (arcs)  , 

2  •   Reachability  Index 

Beachatility  index  is  a  measurement  of  the 
possibilities  to  get  to  a  specified  node,  computed 
ever  all  nodes  of  the  directed  graph.  It  is 
cemputed  with  the  formula: 


=   )   path  to  node  (i)  . 


3-   Debugging 

Debugging   is   the   action   one   takes  to  locate  and 
correct  a  known  or  detected  error  in  a  program. 
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4  .   Tes t in  3 

Testing  is  the  action  to  check  whether  a  program 
meets  its  specifications  and  tc  establish  the 
presence  of  errors  in  it. 

5-   liJi  SJcJL£  2l   &    Program 

Ihe  life  cycle  of  a  program  consists  of  the 
following  phases: 

-  design 

-  Coding 

-  Cebucging 

-  Testing 

-  Production  and  maintenance. 
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III.   MODERN  PROGRAMMING  TECHNIQUES 


Two  recent  developments  in  the  theory  and  practice  of 
software  development  are  addressed  here  as  important  because 
they  are  relevant  not  only  for  the  actual  writing  cf  the 
code  of  the  program,  but  also  to  debugging,  testing,  and 
integrating  software  systems  as  well,  namely  the  advent  of 
modular  and  structured  programming.  The  advantages  cf  these 
technigues  are  chv4ous  for  the  programmer  when  he  develops 
his  program.  Programs  written  using  these  technigues  are 
easier  tc  lead  and  to  understand  as  far  as  the  flew  cf  the 
logic  is  concerned.  Also,  the  tester  can  better  understand 
the  logic  cf  a  program  when  these  technigues  are  employed. 
Furthermore,  it  has  been  proposed  for  structured  programs  tc 
elininate  flowcharts  as  media  cf  communication  [13],  so  it 
is  necessary  tc  understand  how  much  testing,  integration  and 
maintenance   cf  software  are  influenced  by  this  development. 


A.   MCDOIiE  PROGRAMMING 


Modular  programming  is  a  system  to  develop  programs  as  a 
set  cf  interrelated  individual  units  (called  modules)  which 
later  can  be  linked  together  tc  form  a  complete  progratr  [9]. 
Thus  modular  programming  is  not  simply  splitting  up  a 
program  into  several  parts  (subroutines),  but  rather 
dividing  the  software  according  to  the  functions  tc  be 
performed.  Ihe  designer  faces  the  one  crucial  problem  which 
will  deternine  success  or  failure,  namely  to  specify 
ccapletel}'  and  carefully  the  interface  between  the 
individual  modules. 
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Modules  as  individual   program   units   should   have   the 
following  properties: 

(1)  Cre  ncdule  should  perfcrn  only  one  basic  function 

(2)  The  size  of  a  module  should  be  such  that  it  is 
easily  understood  and  contains  ccly  a  moderate 
ancutt  of  code 

(3)  fl  module  should  be  designed  in  such  a  way  that  it 
has  cnly  a  few  control  or  data  paths 

(4)  Cne  ncdule  should  process  only  a  small  amount  of 
data . 

The  design  of  programs  in  this  way  leads  not  cnly  to 
cleaner  a  r.d  more  productive  coding  cut  also  to  easier  and 
ioi€  flexible  testing.  The  advantages  with  respect  to 
debugging  and  testing  show  up  in  several  ways.  Single 
modules  can  he  debugged  and  tested  independently  froi  the 
ether  mcdules  or  the  main  (driver)  program.  Furthermore,  if 
the  modules  are  snail  enough,  extensive  testing  generally 
assumed  as  impossible  with  the  exception  of  very  trivial 
programs,  can  become  manageable.  This  in  turn  leads  tc  more 
reliable  programs.  If  all  modules  of  a  software  project  can 
be  tested  extensively,  a  highly  reliable  program  can  be 
produced.  Even  if  one  falls  short  of  this  goal  -  and  this 
happens  ir  mest  cases  due  to  the  very  large  number  of 
possible  inputs  and  program  paths  -  the  final  prograi  will 
te  more  reliable  and  more  thoroughly  tested  than  a 
ncn-mcdular  program.  The  possibility  of  testing  mcdules 
individually  provides  for  better  (more  eccncnical) 
allocation  of  testing  resources,  because  cne  does  net  have 
tc  wait  until  the  whele  program  has  been  completed.  However, 
to  test  individual  modules,  special  test-rcutines  are  needed 
as  drivers  and  if  ether  modules  must  interact,  dummy  mcdules 
must  be  created  if  tie  real  mcdules  are  not  yet  available  or 
net  yet  tested. 
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One  final  point  in  favour  of  modular  programming  has  to 
be  aade:  Normally,  no  production  program  is  completed  until 
the  day  when  it  is  no  longer  used,  i.e.  every  running 
production  program  has  to  be  maintained  and  adapted  tc  new 
considerations  and  situations.  Because  of  the  simplicity  of 
the  overall  organization  of  modular  programs  this  software 
maintenance  is  alleviated  since  interactions  between  modules 
are  more  easily  understood;  hence,  the  effect  of  program 
changes  is  easier  to  identify.  Also  only  the  modules 
affected  ty  the  chanjge  have  to  be  tested  (together  fcith  the 
main  program  and  interacting  modules) . 


B.   SSR0C1UEII  PROGRAMMING 


Having  ceded  a  program  in  the  atove  described 
modularized  fashion,  there  is  still  room  for  improvement. 
Since  Eijkstra's  famous  letter  to  the  editor  of  the 
Communications  of  tie  ACM  in  which  he  proposed  to  eliminate 
GO-TO  statements  [5]*  the  concept  of  Structured  Programming 
has  evolved  and  led  tc  further  simplification  of  the  coding 
process. 

Simplification  means  here  not  that  the  actual  cede  is 
easier  tc  write  -  although  this  might  be  the  case  tec  for  a 
programmer  who  is  familiar  with  the  concept  and  can  think  in 
these  terms  -  but  the  code  produced  and  the  control  secuence 
of  the  finished  program  is  simpler  than  in  a  nonstructured 
program.  This  simplification  has  been  theoretically 
demonstrated  ty  BoehjJi  and  Jaccpini  as  early  as  1966  [3  1. 
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Although  there  are  as  many  interpretations  of  what 
Structured  Erogramming  is  as  there  are  authors  on  this 
topic,  the  following  features  are  essential  and  common  to 
this  concept: 

(1)  1CP-ICWN  Design,  i.e.  the  design  starts  at  a  very 
general  level  and  proceeds  stepwise  to  the  specific 
and  detailed  tasks 

(2)  Modular  Design 

(3)  Limited  possibilities  to  control  the  logic  flew  of 
the  prograa,  namely  only 

*  seguential 

*  conditional:     IF  -  THEN  -  ELSE 

*  iterative:       DO  -  WHILE 

statements  are  allowed. 

Whereas  the  so  called  block-structured  languages  like 
ALGOL  or  EL/I  lend  themselves  to  this  form  of  coding 
(although  GOTO  statements  are  provided'  by  the  language), 
even  in  ECETEAN  the  implementation  of  some  of  the  basic 
principles  cf  Structured  Programming  is  possible  if  the 
programmer  concerned  with  a  structural  flew  cf  his  program 
cheeses  the  tranching  caused  by  unavoidable  GOTO 
statements  carefully. 

Eaker  [1]  shows  that  the  application  of  Structured 
Programming  combined  with  the  "Chief  Programmer  Team  Method" 
of  organizing  a  software  project  [2]  can  bring  measurable 
improvements  in  software  development,  in  the  coding  as  well 
as  in  the  debugging  and  in  the  testing  stage.  Due  to  the 
fact.that  Structured  Programming  implies  Modular  Programming 
the  same  advantages  hold  here  too,  i.e.  the  software  is 
easier  tc  test  and  tp  maintain  after  release. 
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IV.   THE  PROBLEM  CF  PROGRAM  COMPLEXITY 


The  inpact  of  the  programming  techniques  described  above 
en  the  eccrcaic  development  of  reliable  and  maintainable 
software  is  directly  related  to  the  problem  of  program 
complexity.  Ihere  is  so  far  no  generally  adopted  definition 
of  what  program  complexity  really  means.  The  definition  is 
dependent  cr  the  context  in  which  one  wants  tc  examine 
program  complexity.  Here  complexity  is  defined  as  structural 
properties  cf  a  program  that  affect  the  ability  tc  detect 
errors. 

Onder  the  condition  that  the  structure  cf  a  program  is 
described  by  a  directed  graph,  the  following  criteria  can  be 
used  to  measure  its  complexity: 

1.  Nuaber  cf  ncdes 

2.  Number  cf  arcs 

3.  Number  cf  possible  paths  through  the  program 

4.  Number  cf  source  statements 

5.  Averace  path  length  (source  statements  per  path,  arcs 
per  paths) 

6.  Reachability  index 

7.  Fullness   index  (ratio  cf  actual  to  maximum  number  of 
arcs)  . 

Although  Mills  in  his  contribution  tc  Ref .  8  generates  the 
idea  cf  equating  program  complexity  with  the  difficulty  of 
understanding  a  prpgram  and  justifies  this  approach  with 
".."..the  frustration  of  concocting  and  demolishing  more 
simple  minded  direct  ideas,  such  as  counts  cf  branches,  data 
references,  etc.",  his  approach  does  not  help  to  get  a  real 
measurement   cf   complexity   such  that  one  is  able  tc  lake  a 
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quantitative  stateliest  how  complex  a  program  is.  It  seems 
that  the  important  point  is  tc  relate  program  complexity  to 
the  problem  area  one  pursues.  The  analysis  of  NTDS-Ecgrams 
has  given  insight  in  methods  to  measure  complexity  «ith 
respect  tc  £ictlems  of  program  design  and  testing. 
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V.   ERROR  DETECTION  SIMULATION  MODEL 


A.   GEHEE4I 


A  Software  Error  Detection  Simulation  Model  was 
criginallj  developed  by  T.F.  Green  in  his  M.S.  Thesis  [7] 
and  subsequently  mpdified  by  professor  G.T.  Howard  cf  the 
Naval  Postgraduate  School.  Written  in  FORTRAN  it  was 
designed  tc  rue  en  the  IBM  360/67  computer  of  the  Naval 
Postgraduate  School.  Originally  it  had  been  tested  acainst 
hypothetical  and  actual  programs.  It  was  shewn  that 
siirulaticn  cf  error  detection  was  feasible  and  that 
information  cculd  be  obtained  on  the  relationship  between 
error  detection  and  program  complexity.  Ecwever,  it  was 
necessary  tc  perform  additional  model  feasibility  tests  by 
using  the  model  en  a  large  number  of  actual  programs.  In 
the  process  cf  testing  some  cf  the  original  features  had  to 
be  removed  and  provisions  had  to  be  made  for  cases  of 
program  behaviour  which  were  unexpected  at  the  time  cf  the 
simulation  program  design.  A  detailed  description  of  the 
model  with  its  specific  assumptions  and  capabilities  is 
found  in  Bef.  10,  pg..  IV-5  -  IV-39. 


E.   PRCGSAM  ^PRESENTATION 


The  prerequisite  for  the  use  of  the  simulation  mccel  is 
to  get  the  structure  of  a  program  that  has  tc  be  tested  in 
the   form   cf   a   directed   graph.    A   directed   graph  is  a 
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convenient  meats  tc  show  the  structure  of  programs.  It  is 
suitable  fcr  showing  the  control  flow  in  a  program,  measures 
of  complexity  can  be  derived  from  this  kind  of 
representation.  In  addition,  the  "control  flow  graph"  as 
this  composition  of  structures  is  sometimes  called,  is  also 
very  useful  for  determining  the  execution  time  of  a 
structure  en  a  machine.  This  representation  of  program 
structures  also  simplifies  the  representation  of  large  and 
ccnplex  programs  because  these  programs  can  te  broken  up  in 
logical  segments  (modules,  procedures,  subroutines  etc.), 
and  the  segments  can  be  tested  separately  from  the  ether 
parts  of  tie  program. 


C.   CUBEEKI  S1AT0S  OF  THE  SIMULATION  PEOGBAM 


1 .   In£U t  iari ables 

Ice   following   input   variables  have  to  be  used  for 
the  simulation: 

a.  MINEGI   designates   the   number  of  inputs  within  each 

replication. 

b.  NDMOOI   is   the   number   of   replications   (number  of 

paths}^  within  every  repetition. 

c.  NEEPEI   is   the   number   of   reseedings   with   errors 

(repetitions) . 

d.  MEANLN   designates   the   mean   arc   length  if  the  arc 

lengtiis  are  selected  at  random  by  the  program 
and  are  not  read  in. 
.  e.  MEANEE   designates  the  mean   number   of   instructions 
between  errors. 

f.  N       is  the  number  of  nodes  within  the   structure. 

g.  Input  fcr  the  Adjacency  Matrix  is  dene  in  a  shorthand 
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nctaticn: 

Fcr  every  node  with  the  exception  of  the  last  nodes 
there  is  one  data  card  which  contains  infcrnation 
ahcut  this  node  in  the  following  sequence: 
Ident if icatioo  of  the  node,  number  cf  arcs  emanating 
frcm  this  node,  identification  numbers  of  the  ncdas 
tc  which  the  arcs  go. 

h.  Input   for   the   matrix   of   arc   lengths   (optional) 
siailar  to  that  for  the   adjacency   matrix:   Instead, 
only   as  the  identifiers  for  receiving  nodes  the  pair 
(identifier,  number  of  statements  on  this  arc)  has  to 
be  provided. 

i.  Input  tc  plant  errors  in  arcs  instead  cf  letting  the 
program  seed  them  at  random:  Input  as  for  matrix  of 
arc  length,  but  the  number  cf  errors  en  this  arc  has 
tc  be  specified  instead  of  the  number  cf  statements. 

j.  aCGT   specifies  the  desired  output: 

0  =  Suttiary  output 

1  =  Extensive  output  (NDMOUT  *  NfiEPEI  <  25) 

2-   IILE^t  formats 

Ihe  input  formats  are  as  follows: 

First   data   card:   (615)    aiNPGT,   NOaOOT,  NREPET,  KEAflLN, 
MI5NE5,  K. 

Seccnd    and   fcllcwing   cards:   adjacency   matrix,   (1615); 
followed  ty  delimiter-card:  99  in  columns  4  and  5. 

Input   cards   for   matrix   of   arc   length   (cptional)  :  215, 
7(15, F5.C);   followed  by  delimiter-card:  99  in  columns  4  and 

5..  .. 

Input  to  seed  errors  manually  (cptional)  :  1615; 
delimiter-care:  99  in  columns  4  and  5. 
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Last  data  caid  (output  specification):  15. 

Note  that  all  delimiter  cards  are   not   opticnal. 

3-   I Alii anions 

This  simulation  program  is  currently  restricted  to 
accomodate  a  naximum  number  of  30  nodes.  The  execution  time 
for  simpler  structures  (about  10  -  15  ncdes)  is  within  a 
five  minute  time  limit.  Larger  and  more  complex  structures 
with  more  redes  and  possible  paths  through  the  structure 
reguire  a  30  minute  time  frame  for  the  execution  of  one 
simulated  input  in  1-00  replications  and  100  repetitions. 

An  extension  of  the  limits  of  the  program  to 
accomodate  larger  structures  seems  to  be  impractical  because 
of  the  fast  rise  of  memory  space  and  execution  time 
reguired . 

**•  iI9£I$.®   Listing 

A  listing  of  the  current  error  detection  simulation 
program  as  it  Mas  used  for  the  analysis  of  the  NTDS-crcgrams 
is  found  in  Apcendix  A. 
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VI.   ANALYSIS  OF  NTDS  PBCGEAflS 


GENEEAL 


In  order  tc  demonstrate  the  practicality  of  program 
analysis  using  the  Error  Detection  Simulation  Model,  Naval 
Tactical  Eata  Systems  Programs  have  been  analyzed  by 

1.  describing   the  structure  by  converting  the  programs 
iotc  the  fcrm  of  directed  graphs 

2.  running   these   structures   on   the   error  detection 
simulation  model     and 

3.  evaluating   the   simulation   results  with  respect  to 
measures  of  program  complexity. 


B.   DESIGN  CE  KTDS  P-ECGEAMS 


1«   ^S^i?2ai  Design 

Ibe  design  of  NTDS  programs  is  characterized  by 
Modular  frccram ling,  both  in  general  and  in  detail,  aEd  the 
modular  design  is  a  characteristic  of  the  hardware  as  well. 
Also  the  actual  inplementation  of  every  NIDS  installation 
consists  of  hardware  and  software  building  blocks  that  are 
composed  tc  fit  exactly  the  need  of  each  installation. 

Although  NIDS  programs  are  really   programmed   in   a 
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modular  fashion,  the  term  "module"  does  net  have  the  same 
meaning  as  usual.  Module  usually  refers  to  basic  building 
blocks  that  are  parts  of  the  program,  whereas  NTDS  programs 
are  composed  cf  subsystems.  The  NTDS-"Modules"  in  turn  are 
divided  up  in  parts  which  correspond  to  the 
"mcdule"-def initicn  of  Modular  Programming.  In  NTDS 
terminology  these  parts  are  called  procedures.  NTDS  modules 
perform  complex  tasks  such  as  tracking,  display  etc.  They 
certain  a  iredium  to  large  number  of  dependent  procedures. 
These  procedures  perform  the  basic  functions  intended  in 
Modular  f rcgramming  such  as  checking  track  properties. 
Throughout  this  discussion,  "module"  is  used  as  in  the  NTDS 
system,  namely  as  a  complete  subsystem  fcr  performing 
complex  tasks. 

Ihe  nodular  approach  is  imbedded  in  a  stringent 
hierarchical  systen  -which  is  controlled  by  the  priorities  of 
the  tasks  to  be  performed.  The  levels  cf  hierarchy  are 
applied  tc  the  modules  in  such  a  way  that  only  major 
subprograms  which  are  designed  to  execute  distinctive  tasks 
can  communicate  with  each  ether,  whereas  the  procedures 
within  the  modules  can  only  communicate  according  to  the 
level  of  hierarchy  they  belong  to,  with  the  exception  of 
calls  tc  certain  system  routines. 

2.   CS-J[  l.§ngua.ge 

Ihe  N1ES  programs  are  written  using  the  CS- 1  high 
level  language  compiler  [6].  This  language  has  the  advantage 
that  it  is  well  suited  to  the  application  area/  namely 
tactical  programs  which  run  under  severe  constraints 
regarding  time  and  memory  space  availability.  Tables  are 
searched  in  a  very  effective  way,  and  another  interesting 
feature  is  that  assembly  code  can  be  interspersed  within  the 
high   level   cede   of   the   program.   This   fact   gives   the 
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programmer   a   powerful   means   for  controlling  the  hardware 
which  in  turn  facilitates  the  production  of  effective   code. 


C.   CIEEC1EE  GEAEH  CONSTRUCTION 


In  crder  to  ottain  the  desired  statistics  and  to  analyze 
the  data-  and  control  flow  of  a  single  NTES-program  ,  the 
following  method  has  been  developed  and  used: 

1.  One  complete  module  from  an  existing  and  currently 
operating  NTDS  program  has  been  put  into  the  form  of 
a  directed  graph.  The  module  has  teen  deccnpcsed 
intc  the  procedures  it  contains,  and  every  procedure 
is  treated  separately.  Due  to  the  modular  design 
thcughcut  the  program,  no  logical  difficulties  arise 
here,  because  every  procedure  has  only  one  entrance 
and  cne  exit  point,  i.e.  the  interface  for 
interacting  procedures  within  the  module  is  uniguely 
defined.  Fcr  each  procedure  the  directed  graph  and 
the  adjacency  matrix  have  been  constructed.  As 
quantitative  measurements  the  number  of  nodes,  arcs, 
paths,  loops,  source  statements,  machine 
instructions,  source  statements  per  arc,  and  machine 
instructions  per  arc  have  been  compiled. 

2.  Ihe  same  work  was  done  fcr  randomly  selected 
procedures  from  one  other  important  module  of  the 
same  program  in  order  to  obtain  comparative  results 
and  to  relate  the  reported  number  of  errors  tc  the 
different  modules. 
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3.  Ecr  the  construction  of  the  directed  graphs  and  the 
gathering  of  the  several  statistics  the  following 
assumptions  have  been  made: 

a.  Nodes  are  associated  with 

(.1)  Procedure  entrance  and  exits 

(2)  IF-statements  (decision  points) 

(3)  Points  where  paths  merge 

(4)  Procedure  calls  within  the  module 

(5)  Beginning  and  ending  of  loops 

t.  All  nodes  within  the  module  are  distinct. 
They  have  individually  assigned  numbers  (some 
nodes  are  indicated  as  "dumiy"  nodes)  ,  and 
they  are  counted  only  once,  namely  in  the 
procedure  they  belong  to. 

c.  Entrance  and  exit  nodes  of  a  called  procedure 
are  regarded  as  "transient"  nodes  within  the 
calling  procedure,  and  one  "transient  arc" 
connects  both  transient  nodes.  This 
transient  arc  represents  all  the  arcs  inside 
the  called  procedure.  The  transient  arcs  are 
indicated  in  the  drawings  by  a  dashed  line. 
Transient  nodes  have  either  the  number  of  the 
entrance  node  of  the  called  procedure  or  they 
are  denoted  by  letters  to  distinguish  them 
from  the  original  nodes  of  the  corresponding 
procedure. 

d.  The  Length  of  every  arc  is  indicated  as  the 
number  of  source  statements  or  the  numter  of 
machine  instructions  respectively.  In  the 
analysis  the  number  of  source  statements  has 
been  used  because  programs  are  normally 
written  in  a  high  level  language  and  this  is 
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the  point  where  errors   are   introduced   into 
the  program. 

4.  Normally,  the  numbers  of  both  source  statements  and 
machine  instructions  have  been  counted  in  the  arc 
where  the  statements  appear.  However,  because 
IJ-statements  and  procedure  calls  result  in 
tranching,  they  have  teen  counted  in  the  arc  leading 
to  the  corresponding  node.  Whereas  for  the  counting 
of  machine  instructions,  it  would  be  possible  in  the 
case  of  an  IF-statement  to  split  the  instruction 
seguence  according  to  the  arcs  emanating  froa  the 
decision  point,  this  is  not  feasible  for  the  source 
statement  which  contains  the  elements  of  both  arcs 
eaanating  from  it;  it  cannot  be  split. 

The  structures  obtained  from  both  modules  anc  the 
compiled  statistics  are  found  in  Appendix  E.  The  following 
figure  shews  hew  to  read  the  structure  diagrams: 


Procedure  Entrance 

IK       2   Source  Stmts.)     .,. 

4  Machine  Instr.  )  on  tnis  arc 


Transient  nodes 
and  arc 


Dummy  node 

/cf' 

Procedure  Exit 


figure  2  -   EXAMPLE  OF  PROGBAM  SIEUCTaSr 
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D.   EEROB  DETECTION  SIMULATION  ON  THESE  STB0CTU3ES 


The  structures  which  were  converted  intc  directed  graphs 
for  Module  One  were  screened  tc  determine  their  suitability 
for  errcr  detection  simulation.  It  would  have  teen 
desirable  to  select  a  random  sample  of  the  structures. 
However,  it  was  necessary  to  choose  structures  which  wculd 
net  require  excessive  amounts  of  memory  space  and  CPU  time 
during  the  simulation.  In  addition,  the  structures  were  to 
have  at  least  two  or  more  paths.  In  the  case  of  Module  Two 
it  was  ieasitle  tp  use  a  random  sample  because  a  high 
percentage  of  the  structures  fell  within  the  memory  space 
and  the  CEU  time  limitations  of  the  model. 

Ihe  input  data  for  the  simulation  were  taken  from  the 
actual  programs,  including  the  number  of  source  statements 
for  every  arc.  The  recorded  number  of  errors  per  module  was 
used  tc  calculate  the  mean  number  of  instructions  between 
errors,  which  is  used  for  seeding  errors  in  the  simulation 
model.  Seeding  the  errors  was  done  randomly  by  the 
simulation  program.  However,  it  was  provided  that  no  errors 
were  seeded  at  arcs  containing  zero  instructions  (ccntrol 
arcs)  . 

The  simulation  was  run  with  one  input,  100  replications 
and  100  repetitions  (reseedings) ,  and  the  average  number  of 
errors  fcund  by  o.ne  input  was  obtained.  Although  scire  of 
the  structures  were  small,  and  a  higher  number  of 
repetitions  and  replications  could  have  been  run,  the  same 
simulation  parameters  were  used  for  each  structure  in  order 
to  obtain  ccatarable  results. 
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E.   RESUIIS  CI    THE  ANALYSIS 


Prom  th€  average  of  errors  found  by  one  input  in  each 
procedure  the  average  percentage  of  errors  fcund  against  the 
errors  expected  within  the  procedure  was  obtained.  These 
results  were  plotted  against  various  complexity  measures, 
e.g.  the  number  of  paths.  Although  the  results  varied 
somewhat  between  the  modules,  it  was  possible  to  establish 
relationships  between  structural  properties  and  error 
detection  capabilities. 

The  differences  in  results  between  modules  can  be  traced 
to  several  factors: 

1.  Different  sample  sizes: 

Ercm  Module  One  32  procedures  were  used,  16 
procedures  were  randomly  selected  frcm  Module  Two. 

2.  The  different  size  of  the  modules: 

Module  Cne  had  97,  and  Module  Two  had  155  procedures. 

3.  Differences  in  program  design  and  programming  style: 
Module  Two  was  modularized  to  a   much   larger   extent 
than   Module   One.    It  was  hard  to  find  a  sufficient 
nuiber  of  paths  within  randomly   selected   procedures 
of  Mcdule  Two. 

4.  Different  number  of  reported  errors: 

Although  Mcdule  Two  was  1.6  times  larger  than  Module 
One,  it  had  only  about  two-thirds  the  number  of 
errors. 

The  following  diagrams  show  the  percentage  of  average 
errors  fcund  against  the  expected  number  of  errors  fcr  the 
structures  cf  both  modules. 
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Ihe  curves     shewn     represent  exponential 

approximations  tc  the  datapcints  according   to  the   formula 

y=a*e**  (-b*x)  which  was  found  to  represent  the  relationship 

best.   A  least  Sguare  fit  was  used. 

All  diagrams  show  seme  relationship  between  error 
detection  and  complexity.  Module  One  with  its  larger  sample 
size  shews  this  relationship  more  than  Module  Two  for  the 
number  of  paths.  This  seems  logical  because  a  large  number 
of  paths  reduces  the  ability  to  detect  errors  in  a  program. 
It  appears  that  the  number  of  paths  could  be  used  as  a 
measure  of  program  complexity  for  design  and  testing 
purposes . 

In  order  tc  rank  the  approximations,  a  sguared  error 
factor  has  been  computed  for  every  complexity  measure  as 
fellows: 

Error  Factor 
Mod.  1   Mod.  2 


Nodes 
Arcs 
Eaths 
S. stmts. 


7337 

4430 

6841 

3933 

4995 

4666 

6575 

1808 

This  computation  sho-ws  that  for  Module  One  the  number  of 
paths  is  the  best  approximated  complexity  measure  ty  the 
method  used.  Another  interesting  aspect  found  was  the  well 
approximated  relationship  between  percentage  of  errors  found 
and  the  nuater  of  source  statements  in  module  Two. 
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VII.   OSE  OF  THE  RESULTS 


A.   AILS    ICE  SCFTW2RE  DEVELOPMENT 


This  methcd  cf  program  analysis  provides  the  software 
manager  with  information  for  selecting  structures  easily  in 
the  design  process.  He  can  choose  the  least  complex 
structure  which  Mill  satisfy  project  requirements. 
Furthermore,  after  a  project  has  been  coded  and  is  due  for 
testing,  he  can  make  realistic  '  assessments  concerning  the 
effort  which  will  be  needed  for  program  testing  by 
considering  factors  such  as 

1.  expected  complexity  of  the  project 

2.  chcice  cf  the  programming  techniques  used 

3.  organizatior  and  experience  of  the  programming  team 

4.  available  manpower   and   computer   time   for   testing 
purposes. 


B.   FDTDEF  tfCEK 


The  analysis  dene  on  the  N1DS  programs  and  the  results 
obtained  fcr  the  measurement  of  program  complexity 
represents  a  modest  contribution  to  the  field  of  software 
engineering.  But  being  far  from  complete  or  exhaustive  the 
following  steps  should  be  taken  in  order  to  obtain 
additional  validation  of  the  analysis  process. 
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1  •      Iu.£*J?€r   £VJ luat  i°Jl   2j£    NTDS-Modules 

Additional  NIDS  modules  should  be  evaluated  in  order 
to  obtain  larger  sample  sizes.  It  is  realized  that  the 
evaluation  process  for  the  important  modules  is  verj  time 
consuming.  However,  the  more  important  modules  are  used 
mere  freguentlj  and  will,  in  most  cases,  have  a  longer  error 
history,  which  will  provide  valuable  data  for  comparison 
with  siaulaticn  results. 

2  •   I va lua ticn  of  structured  .procjr  a  ms 

It  would  be  of  interest  in  this  respect  to  compare 
the  evaluaticr  of  the  NTDS-prccedures  with  procedures  that 
perform  the  same  functions  but  are  rewritten  and  converted 
into  a  structured  programmed  form.  It  is  expected  that  the 
structured  programs  would  perform  better  with  respect  to 
error  detecticn. 
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VIII.   SUMMARY  AND  CONCLUSIONS 


A  method  to  define  and  analyze  program  structures  has 
teen  presented.  All  measurements  obtained  were  based  en  the 
description  cf  the  program  structure  in  the  fens  cf  a 
directed  crajh  and  the  use  of  the  error  detection  simulation 
model.  Ihis  method  has  been  used  to  analyze  the  procedures 
from  two  KITS  modules.  It  was  beyond  the  scope  cf  this 
effort  to  obtain  comparative  results  between  this  experiment 
and  the  actual  error  history  cf  the  programs.  However,  it 
was  possible  tc  obtain  an  initial  quantitative  assessment  of 
measures  cf  complexity. 

By  using  this  method  to  check  program  structures  in  the 
design  phases  it  should  be  possible  to  produce  programs  with 
structures  that  are  less  complex  and  therefore  easier  and 
more  economical  to  test  and  maintain.  Also  the  method  could 
be  used  during  the  test  phase  as  a  means  cf  assigning  test 
resources . 
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APPENDIX  A 


ERROB  DETECTION  SIMULATION  ESCGEAM 


The  program  listed  on  the  following  pages  shews  the 
Software  Errcr  Detection  Simulation  Prograir  as  used  in  the 
analysis  cf  the  NTDS  procedures  in  the  version  currert  in 
May  1976.  although  carefully  tested  the  program  should  not 
he  regarded  as  a  final  release. 
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APPENDIX  B 


IIST  CF  EVALUATED  PROGRAM  STRUCTURES 


This  list  gives  all  the  statistical  data  gathered  frcrc 
the  conversion  of  the  procedures  of  the  NIES  modules  into 
the  fori  of  directed  graphs.  The  abbreviations  r€ad  as 
fellows: 

PNR  Procedure  number  within  the  module 

N  Number  cf  nodes  (including  transient  nodes) 

A  Nuirber  of  arcs  (including  transient  arcs) 

P  Nunber  of  caths 

L  Nunber  cf  loops 

Ss  Number  of  source  statements 

Mi  Number  of  machine  instructions 

SA  Scuice  stmts. /arc 


MA     Machine  instr./arc 
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APPENDIX  C 


DIRECTED  GEAPHS 


Cd  the  following  pages  the  structures  of  all  the 
procedures  are  listed  that  were  used  as  input  data  for  the 
Error  Detection  Simulation  Model.  In  addition  tc  the 
complexity  measures  used  also  listed  are  the  results 
ottained  frcm  the  simulation,  the  average  number  cf  errors 
found  with  1  input,  100  replications  and  100  repetitions, 
and  the  percentage  of  expected  errors  detected. 

Differently  tc  the  sample  structure  shown  in  Fig. 
2,  the  number  cf  statements  is  indicated  in  the  following 
graphs  only  fcr  arcs  with  ncnzerc  instructions. 

The  count  for  the  number  of  nodes  and  the  numher  of 
arcs  includes  the  transient  nodes  (designated  by  letters) 
and  the  transient  arcs  (dashed  lines)  because  they  must  be 
included  intc  the  inputs  for  the  Error  Detection  Simulation 
Program. 
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Module:   1 


Procedure  No.:   2 


Nunber  of  nodes:  14 

Numfcer  of  arcs:  23 

Hunter  of  paths:  26 

Nuafcer  ci  source  stmts.:  37 

Average  errcr  found:  0.3144 

Percentage  ericrs  found:  17.84 
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Module:   1 


Procedure  No 


Nuifcer  of  nodes:  13 

Numrer  of  arcs:  14 

Nuiifcer  of  paths:  3 

Nuirter  cf  source  stmts.:  10 

Average  errcr  found:  0.2523 

Percentage  errors  found:  52.98 


55 


Module:   1 


Procedure  No. 


11 


Nuiifcer  of  nodes:  6 

NuiEter  cf  arcs:  8 

Nunfcer  of  paths:  5 

Nuiiher  cf  sctrce  stmts.:  8 

Average  errcr  found:  0.1974 

Percentace  eircrs  found:  51.82 
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Module:   1 


Procedure  No.;   1U 


4/11 


2/7 


3/6 


Nunter  of  nodes:  6 

Nuifcer  of  arcs:  7 

Nuitber  of  paths:  4 

NuiEter  cf  source  stmts.:  9 

Average  errcr  found:  0.2586 

Sercentace  eircrs  found:  60.34 
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Module:   1 


Procedure  No 


19 


Hunter  of  nodes:  19 

Number  of  arcs:  26 

Nuafcer  of  paths:  7 

Number  cf  source  stmts.:  45 

Average  errcr  found:  0.2885 

Percentage  errcrs  found:  27.54 
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Module:   1 


Procedure  No 


t/. 


Hunter  of  nodes:  25 

Nuniter  of  arcs:  30 

Nunter  of  paths:  11 

Nunter  of  source  stmts.:  30 

Average  errcr  found:  0.4105 

Percentage  errors  found:  28.74 
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Mcdule 


Procedure  No. 


25 


1/1 


Nunfcer  of  nodes:  12 

Number  of  arcs:  12 

Nunfcer  of  paths:  2 

Nunfcer  cf  sctice  stmts.:  8 

Average  errci  found:  0.2324 

Percentage  errcrs  found:  61.01 
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Module:   1 


Procedure  No.  :   26 


3/H 


Nunfcer  of  nodes:  17 

Number  of  arcs:  19 

Nunter  of  paths:  4 

Number  cf  source  stmts.:  32 

Average  error  found:  0.6400 

Fercentace  errors  found:  42.00 
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Module:   1 


Procedure  No 


29 


Nucfcer  of  nodes: 
Hunter  of  arcs: 
Nunter  of  paths: 
Hunter  of  sctrce  stmts.: 
Average  errci  found: 
Eercentace  errors  found: 
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Mcdule:   1 


Procedure  No. 


30 


Hunter  of  nodes:  7 

Number  of  arcs:  10 

Number  of  paths:  5 

Number  cf  scuce  stmts.:  10 

Average  errcr  found:  0.1649 

Eercentace  errors  found:  34.63 
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Module:   1 


Procedure  No. :   34 


Hunter  of  nodes:  16 

Nuirter  of  arcs:  17 

Nuirter  of  paths:  3 

Nuirter  of  source  stmts.:  5 

Average  eircr  found:  0-5465 

Eercentace  errors  found:  76.51 
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Module:   1 


Procedure  No.  :   35 


Nuafcer  of  ncdes:  14 

Number  cf  arcs:  17 

Nuirter  of  paths:  3 

Number  of  source  stmts.:  14 

Average  errcr  found:  0.3576 

Fercentace  eircrs  found:  53.64 
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flcdule:   1 


Procedure  No 


36 


Nunfcer  of  ncces: 
Number  cf  arcs: 
Nunfcer  of  paths: 
Nunfcer  cf  source  stmts.: 
Average  errcr  found: 
Percentage  errors  found: 
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Module:   1 


Procedure  No 


39 


1/1 


Nunfcer  of  cedes:  17 

Numrer  of  arcs:  25 

Nunrer  of  paths:  10 

Number  cf  scirce  stmts.:  17 

Average  eircr  found:  0-2637 

fercentace  errors  found:  32.57 
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Module:   1 


Procedure  Ho. :   44 


Nunter  of  nodes:  27 

Number  cf  arcs:  30 

Hunter  of  paths:  7 

Nunter  cf  scurce  stmts.:  21 

Average  errcr  found:  0.3554 

Percentage  eircrs  found:  35.54 
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Module:   1 


Procedure  No 


47 


Nunber  of  nodes:  19 

Number  of  arcs:  20 

Number  of  paths:  4 

Number  cf  source  stmts.:  12 

Average  errcr  found:  0.4231 

Eercentage  errors  found:  74.04 
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Module:   1 


Procedure  No. :   48 


Nuitcr  of  noces:  23 

Nunfcer  of  arcs:  26 

Nuat€r  of  paths:  7 

Number  cf  source  stmts.:  13 

Average  errcr  found:  0.3287 

Eercentace  errors  found:  53. 1C 
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Module:       1 


Procedure    No.  :       49 


Nuufcer    of    nodes:  15 

N Ulster   of   arcs:  18 

Nuiiter  of  paths:  7 

Hunter  of  source  stmts.:  19 

Average  errci  found:  0.2217 

Percentage  errors  found:  24.50 
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Module:   1 


Procedure  No 


53 


Nun ter  of  nodes:  1 1 

Number  of  arcs:  18 

Number  of  paths:  9 

Number  of  source  stmts.:  11 

Average  error  found:  0.1876 

Eercentace  errors  found:  35.81 
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Kcdule:   1 


Procedure  No.:   57 


Nunber  of  ncces:  30 

Number  of  arcs:  36 

Nunber  of  paths:  14 

Number  cf  sctrce  stmts.:  26 

Average  errcr  found:  0.2910 

Eercentage  errors  found:  23.50 
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Module;   1 


Procedure  No.:   60 


Nuirter  of  ncces:  28 

Number  of  arcs:  37 

Nunter  of  paths:  18 

Nuiiber  cf  source  stmts.:  24 

Average  errcr  found:  0.3336 

Eercentace  eircrs  found:  29.19 
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Module:   1 


Procedure  No, 


75 


Hunter  of  ncces: 
Nuaifcer  ci  arcs: 
Nunfcer  of  paths: 
Number  cf  source  stmts. 
Average  error  found: 
Percentage  errors  found 
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Module:   1 


Procedure  No.  :   76 


Nunfcer  of  nodes:  15 

Number  of  arcs:  19 

Nuater  of  paths:  5 

Nuater  of  source  stmts.:  20 

Average  error  found:  0.3893 

fercentace  errors  found:  U0.88 
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Module:   1 


Procedure  No.  :   77 


Hunter  of  nodes:  17 

Number  of  aics:  20 

Nuuter  of  paths:  9 

number  cf  source  stmts.:  10 

Average  errci  found:  0-2425 

Percentage  errors  found:  50.93 
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Module 


Nun  ker  of  node s: 
Numfcer  of  arcs: 
Nuiirer  oi  paths: 
Number  of  sccrce  stmts.: 
Average  errci  found: 
Percentage  errors  found: 


Procedure  No 


19 
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flcdule:   1 


Procedure  No.  :   81 


Nunter  of  nodes:  8 

Nunber  of  arcs:  9 

Nuirfcer  of  paths:  3 

Nunter  ct   source  stmts.:  7 

Average  errcr  found:  0.1449 

Eercentaae  encrs  found:  43.47 
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Module:   1 


Procedure  No. :   86 


Nuafcer  of  ncces:  18 

Nunter  of  arcs:  23 

Nunrer  of  paths:  13 

Number  c£  source  stmts.:  22 

Average  eircr  found:  0.3370 

Eercentag€  eircrs  found:  32.17 
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Module:       1 


Procedure    No.:       87 


2/11 


Hunter  of  nodes:  21 

Number  of  arcs:  22 

Number  of  paths:  6 

Number  cf  source  stmts.:  25 

Average  error  found:  0-5029 

Fercentace  eircrs  found:  42.24 


Mcdule:   1 


Procedure  No. :   9 1 


Nunter  of  nodes: 
Nunter  of  arcs: 
Hunter  of  paths: 
Nunter  of  source  stmts.: 
Average  error  found: 
Percentage  errors  found: 


82 


Module:   1 


Procedure  Mo. :   92 


Nuafcer  of  eccgs:  5 

Numfcer  of  arcs:  6 

Nuiiter  of  paths:  3 

Hunter  of  sccice  stmts.:  9 

Average  error  found:  0.1837 

Percentage  errors  found:  42.86 
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Module:   1 


Procedure  No. :   93 


Nunrer  of  nodes:  25 

Nuttter  of  arcs:  34 

Numfcer  of  paths:  12 

Nuaoter  of  source  stmts.:  34 

Average  error  found:  0.3972 

Eercentac^  eircrs  found:  24.53 
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Module:   1 


Procedure  No.  :   95 


Nunter  of  nodes:  18 

Nunher  of  arcs:  27 

Nunter  of  faths:  10 

Number  cf  source  stmts.:  19 

average  errcr  found:  0.1822 

Eercentace  errors  found:  20.14 
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Module:   2 


Procedure  No. :   15 


Nuttber  of  ncces:  11 

Nunster  of  arcs:  13 

Nunfcer  of  paths:  5 

Nuitcer  cf  sctice  stmts.:  12 

Average  errci  found:  0-0836 

Eercentace  eircrs  found:  35. 5S 
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ilcdule 


Procedure  No. :   23 


Nuirfcer  of  ncces:  10 

Nuufcer  cf  arcs:  11 

Hunter  of  paths:  3 

Number  cf  scurce  stmts.:  12 

Average  errci  found:  0.1592 

Percentage  eircrs  found:  67.66 
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Mcdule:   2 


Procedure  No.:   UO 


Hunter  of  ncces:  22 

Nunber  cf  arcs:  27 

Nuafcer  of  paths:  5 

Nuifcer  cf  scuice  stmts.:  14 

Average  errcr  fcund:  0.2018 

Percentage  errcrs  found:  73.51 


38 


Module: 


Procedure  No. : 


41 


Nuaber  of  noces:  10 

Nuuter  of  arcs:  14 

Hunter  of  paths:  12 

Nunrer  cf  source  stmts.:  13 

Average  errci  found:  0.1554 

Percentage  errors  found:  60.96 
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Module:   2 


Procedure  No. :   46 


Nunfcer  of  ncces: 
Number  of  arcs: 
Number  of  paths: 
Number  of  source  stmts.: 
Average  error  found: 
Percentage  errors  found: 
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Module 


Procedure  No.:   47 


Hunter  of  ncces: 
Number  of  arcs: 
Hunter  of  paths: 
Hunter  cf  sctice  stints. 
Average  errcr  found: 
Percentage  eircrs  found 
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Module: 


Procedure  No. :   48 


Hunter  of  cedes:  16 

Number  of  arcs:  21 

Hunter  of  paths:  14 

Nuater  cf  sciice  stm.ts.:  17 

Average  errcr  found:  0.1580 

Fercentace  eircrs  found:  47.40 
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Module:   2 


Procedure  No. :   73 


Sumter  of  Dcdes:  18 

Hunter  of  arcs:  22 

Nuiiter  of  catbs:  10 

Nuttier  cf  source  stmts.:  34 

Average  errci  found:  0.1885 

Percentage  errors  found:  28.28 
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Module 


Procedure  No. 


79 


Hunter  of  nodes:  13 

Hunter  of  arcs:  14 

Nunter  of  paths:  5 

Nutter  of  scuce  stmts.:  11 

Average  errci  found:  0.1379 

Eercentace  eircrs  found:  63.94 
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flcdule:   2 


Procedure  No 


82 


Nuiirer  cf  ncdes:  23 

Nuafcer  cf  arcs:  24 

Nuuher  of  paths:  2 

Nuater  cf  scirce  stunts.:  3  5 

Average  errcr  found:  0.1130 

Eercentace  eircrs  found:  16.47 
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Module 


Procedure  No. 


86 


Hunter  of  nodes:  30 

Nunter  of  arcs:  34 

Hunter  of  paths:  6 

Nutter  of  source  stmts.:  33 

Average  errcr  found:  0.2042 

Percentage  errors  found:  31.56 
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Mcdule:   2 


Procedure  No. :   90 


1/1 


Nunfcer  of  ncces:  13 

Numfcer  cf  arcs:  18 

Nunter  of  paths:  8 

Nuaher  cf  sccrce  stmts.:  23 

Average  errci  found:  0.0958 

Eercentage  errors  found:  21.24 
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Med  ule 


Procedure  No 


99 


Nuirher  of  nodes:  25 

Nurcter  of  arcs:  30 

flunier  of  paths:  10 

Hunter  of  sctrce  stm«ts.:  23 

Average  errcr  found:  0.1513 

Eercentace  errors  found:  33.55 
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Mcdule:   2 


Procedure  No.:   122 


Nunfcer  of  nodes:  18 

Numter  cf  arcs:  21 

Nuiter  of  paths:  6 

Nuirer  cf  sctrce  stmts.:  20 

Average  errci  found:  0.1686 

Percentace  errors  found:  U2.99 
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Module:   2 


Procedure  No. :   137 


Nunter  of  ncces:  13 

Number  of  arcs:  17 

Nunter  of  paths:  11 

Number  cf  source  stmts.:  23 

Average  errcr  found:  0.1178 

Eercentage  errors  found:  26.12 
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Mcdule, 


Procedure  No.:   149 


Hunter  of  nodes:  18 

Nunrer  of  arcs:  25 

Nuarer  of  paths:  9 

Number  of  source  stmts.:  35 

Average  errcr  found:  0.2357 

Eercentace  errors  found:  34.34 
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